1. Field of the Invention
The present invention relates to a technique of suppressing the deterioration of image quality due to a foreign substance adhering to the surface of an optical low-pass filter or the like in an image capturing apparatus using an image sensor such as a CCD or CMOS sensor.
2. Description of the Related Art
Recently, many image capturing apparatuses have appeared on the market, which generate an image signal using an image sensor such as a CCD and records it as data, like digital cameras and digital video cameras. A digital camera eliminates the necessity of a photosensitive film, which has been conventionally used as a recording medium, and records an image as data on a data recording medium such as a semiconductor memory card or a hard disk drive instead of such a film. These data recording media allow repetitive write and erase operations unlike films, and hence can reduce the consumable expenditure. That is, such media are very useful.
In general, a digital camera is equipped with an LCD (Liquid Crystal Display) monitor capable of displaying shot images as needed and a detachable large-capacity storage device.
Using the digital camera comprising these two devices can eliminate the necessity of a film, which is a recording medium conventionally used as a consumable, and allows the user to immediately check a shot image on the spot by displaying it on the LCD monitor device. It is therefore possible to erase any unsatisfactory image data on the spot or perform re-shooting as needed. That is, as compared with silver halide cameras using films, digital cameras have remarkably improved the efficiency of shooting operation.
Owing to such convenience and technical innovation such as an increase in the resolution of image sensors, the application range of digital cameras has been expanded. Recently, there have been available many lens interchangeable digital cameras such as single-lens reflex cameras.
In a digital camera, however, a foreign substance such as dust or mote (to be simply referred to as dust hereinafter) sometimes adheres to an image sensor protective glass, the surface of an optical filter or the like, or an optical system which are placed on the shooting optical axis in front of an image sensor (which will be generically referred to as an image sensor optical component hereinafter). When dust adheres to an image sensor optical component in this manner, the dust blocks light. As a result, for example, the corresponding portion cannot be shot. That is, the quality of the shot image deteriorates.
In cameras using silver halide films as well as in digital cameras, dust on a film is also captured on an image. However, the film changes after every shot, and hence similar dust is very rarely captured on all frames.
In contrast, the image sensor of the digital camera does not move, and the camera performs shooting using the same image sensor. For this reason, once dust adheres to an image sensor optical component, the dust is captured on many frames (shot images) in the same manner. A lens interchangeable digital camera, in particular, has a problem that dust tends to enter the camera at the time of lens interchange.
The user of the camera therefore must always care about the adhesion of dust to an image sensor optical component, and spends much effort to check and clean dust. Since an image sensor, in particular, is placed relatively deep inside the camera, it is not easy to clean or check dust.
Dust easily enters a lens interchangeable digital camera when a lens is attached/detached. Furthermore, in many lens interchangeable digital cameras, a focal plane shutter is placed immediately before an image sensor, and hence dust easily adheres to an image sensor optical component.
Since such dust generally adheres onto a protective glass or an optical filter instead of the surface of an image sensor, the dust is imaged in different states depending on the aperture value of a shooting lens or the distance from the pupil of the shooting lens to the capturing plane. That is, as the aperture value approaches the open F-number, a shot image blurs, and hence even the adhesion of small dust has almost no influence on the image. In contrast, as the aperture value increases, such dust is clearly imaged, and hence affects the image.
There is known a method of making dust less noticeable by preparing in advance an image of only dust on an image sensor, which is obtained by shooting a white wall or the like while the lens is set at a large aperture value, and using the image in combination with a general shot image (see Japanese Patent Laid-Open No. 2004-222231). This method, however, is cumbersome because the user must always be conscious of the correspondence between the image shot for dust detection and shot images to be associated with it.
It is therefore conceivable to hold information indicating the position of dust, which is acquired by shooting a white wall or the like, and attaching the information indicating the position or size of dust to image data obtained by general shooting operation. For example, using a separately prepared image processing apparatus makes it possible to analyze the dust position on the shot data from the attached information of the position of dust and make the image of the dust less noticeable by interpolating the analyzed area with surrounding pixels.
It is however difficult to make all dust images captured on images less noticeable by such processing. This is because, when information indicating the position of dust is to be acquired by shooting a white wall using a digital camera, the camera sometimes fails to acquire the information indicating the position of dust due to the influence of limb darkening or the like. In this case, since the dust is not registered in the list of dust positions and sizes, it is impossible to make the dust less noticeable by analysis and interpolation processing using the above image processing apparatus.
In addition, the list of dust positions and sizes increases in size as the number of dust particles registered increases. There is some limitation on the number of dust particles which can be registered in the list. That is, if an infinite number of dust particles adhere to the image sensor, it is impossible to register all the dust particles in the list. In such a case as well, there is dust whose position or size cannot be registered in the list, it is impossible to make the dust which is not registered less noticeable using analysis and interpolation processing using the above image processing apparatus.
As described above, any dust which is not registered in the above dust list cannot be made less noticeable by image processing using the dust list. That is, the user needs to manually designate the position of such dust on a shot image by himself/herself and make the dust less noticeable by performing a repair process or the like. If, however, there are a plurality of shot images, the user needs to perform such a manual repair process for all the images. This greatly increases the workload on the user.